Improving Question Answering Sentence Selection by Rank Propagation

Open-domain Question Answering (QA) systems typically leverage an answer selection component to rank candidate answer sentences based on how likely they will contain the answer to a given question. This component plays a crucial rule in the QA system as it usually dictates how downstream processing modules (e.g., answer extraction) retain and present answers to users. Most existing works in this field use machine learning techniques that captures relations between question and answer sentences to train models for ranking answer candidates. In this paper, we introduce RankProp, a method that can be used as a post processing step for improving the quality of the ranking results from classifiers. Based on the assumption that similar sentences should be ranked closer and answer sentences that ranked at the top by classifiers are more likely to contain the answer, our algorithm makes use of similarities among the answer sentences and propagate this information. It takes ranking scores and similarities as input and uses convex optimization to perform further ranking adjustment. Similarities can be evaluated through word embedding techniques. In addition, if the expected answer type can be generated from the answer type prediction module, our algorithm will also incorporate this information together with the entity types extracted from the candidate answers to further improve the quality of the ranking. Experimental results on Jacana-QA system demonstrate that our method generally improve the performance of ranking module by 5% on average. Moreover, this RankProp algorithm is generic in the sense that it does not make any assumption about how the overall QA system works. Therefore, in principle, it can be inserted into an existing QA system to improve its performance without modifying any other components.